Method and system for determining a relevancy parameter for content item

ABSTRACT

A method of determining a relevancy parameter for a digital content item and a system for implementing the method. The digital content item is originated from a content channel associated with a recommendation system. The method is executable by the server. The method comprises: identifying a pool of users associated with the content channel, a given user of the pool of users being associated with the content channel. The method comprises using the pool of users to explore and predict a relevancy parameter. The relevancy parameter is then used for predicting relevancy parameter of the digital content item for a user outside of the pool of users based on the user interactions of the first user.

CROSS-REFERENCE

The present application claims priority from Russian Patent ApplicationNo. 2018132716, entitled “Method and System for Determining a RelevancyParameter for Content Item”, filed Sep. 14, 2018, the entirety of whichis incorporated herein by reference.

FIELD

The present technology relates to recommendation systems in general andspecifically to a method and a system for determining a relevancyparameter for a content item.

BACKGROUND

Various global or local communication networks (the Internet, the WorldWide Web, local area networks and the like) offer a user a vast amountof information. The information includes a multitude of contextualtopics, such as but not limited to, news and current affairs, maps,company information, financial information and resources, trafficinformation, games and entertainment related information. Users use avariety of client devices (desktop, laptop, notebook, smartphone,tablets and the like) to have access to rich content (like images,audio, video, animation, and other multimedia content from suchnetworks).

The volume of available information through various Internet resourceshas grown exponentially in the past couple of years. Several solutionshave been developed in order to allow a typical user to find theinformation that the user is looking for. One example of such a solutionis a search engine. Examples of the search engines include GOOGLE™search engine, YANDEX™ search engine, YAHOO!™ search engine and thelike. The user can access the search engine interface and submit asearch query associated with the information that the user is desirousof locating on the Internet. In response to the search query, the searchengine provides a ranked list of search results. The ranked list ofsearch results is generated based on various ranking algorithms employedby the particular search engine that is being used by the userperforming the search. The overall goal of such ranking algorithms is topresent the most relevant search results at the top of the ranked list,while less relevant search results would be positioned on less prominentpositions of the ranked list of search results (with the least relevantsearch results being located towards the bottom of the ranked list ofsearch results).

The search engines typically provide a good search tool for a searchquery that the user knows apriori that she/he wants to search. In otherwords, if the user is interested in obtaining information about the mostpopular destinations in Italy (i.e. a known search topic), the usercould submit a search query: “The most popular destinations in Spain?”The search engine will then present a ranked list of Internet resourcesthat are potentially relevant to the search query. The user can thenbrowse the ranked list of search results in order to obtain informationshe/he is interested in as it related to places to visit in Spain. Ifthe user, for whatever reason, is not satisfied with the uncoveredsearch results, the user can re-run the search, for example, with a morefocused search query, such as “The most popular destinations in Spain inthe summer?”, “The most popular destinations in the South of Spain?”,“The most popular destinations for a culinary getaway in Spain?”.

There is another approach that has been proposed for allowing the userto discover content and, more precisely, to allow for discovering and/orrecommending content that the user may not be expressly interested insearching for. In a sense, such systems recommend content to the userwithout an express search request based on explicit or implicitinterests of the user.

An example of such a system is a FLIPBOARD™ recommendation system, whichsystem aggregates and recommends content from various social networks.The FLIPBOARD recommendation system presents the uncovered content in a“magazine style” format, where the user can “flip” through the pageswith the recommended/aggregated content. The recommendation systemcollects content from social media and other websites, presents it inmagazine format, and allows users to “flip” through theirsocial-networking feeds and feeds from websites that have partnered withthe company, effectively “recommending” content to the user even thoughthe user may not have expressly expressed her/his desire in theparticular content.

Another example of the recommendation system is YANDEX.ZEN™recommendation system. The Yandex.Zen recommendation system recommendsdigital content (such as articles, news, and video in a personalizedfeed on the Yandex.Browser start screen). As the user browses theYandex.Zen server recommended content, the server acquires explicit (byasking whether the user likes to see more of such content in the user'sfeed) or implicit (by observing user content interactions) feedback.Using the user feedback, the Yandex.Zen server continuously improves thecontent recommendations presented to the given user.

US 2012/0209907 published on Aug. 16, 2012 to Andrews et al discloses acontent aggregation and distribution service, which can execute in acloud computing environment, provides content based on a broadcastuser's topics of interest to a subscriber user based on the context ofthe subscriber. An example of a broadcast user is a celebrity. Contentis automatically gathered about the broadcast user's designated topicsof interest from online resources, and filtered and distributed based ona context of the subscriber. Some examples of online resources arewebsites, social networking sites, and purchase transaction systems. Anexample of broadcast content is a recommendation which may have beenentered directly to the service or posted by the celebrity in his or hersocial networking account. Both the broadcast user and the subscribercan control respectively the distribution and reception of content withsubscription settings. For examples, the settings may set limitationswith respect to topics, contexts, and subscriber profile data.

US 2017/103343 published on Apr. 13, 2017 to Google Inc. disclosesmechanisms for recommending content items based on topics are provided.In some implementations, a method for recommending content items isprovided that includes: determining a plurality of accessed contentitems associated with a user, wherein each of the plurality of contentitems is associated with a plurality of topics; determining theplurality of topics associated with each of the plurality of accessedcontent items; generating a model of user interests based on theplurality of topics, wherein the model implements a machine learningtechnique to determine a plurality of weights for assigning to each ofthe plurality of topics; applying the model to determine, for aplurality of content items, a probability that the user would watch acontent item of the plurality of content items; ranking the plurality ofcontent items based on the determined probabilities; and selecting asubset of the plurality of content items to recommend to the user basedon the ranked content items.

CN 103559262 that has a notification of grant of patent rights datedJul. 29, 2016 to Beijing University pf Posts and Communicationsdiscloses a community-based author and academic paper recommendingsystem and a recommending method. A double-layer quotation networkconsisting of an author layer and an academic paper layer is formed byutilizing a quotation relation between an author and the academic paperand the community information, then a user interesting model isestablished according to a historic behavior record of the user and theacademic paper set read by the user, finally the user demand is analyzedaccording to the obtained double-layer quotation network and the userinteresting model, and the author and academic paper thereof can berecommended to the user. The system is provided with an academic papercapturing module, an academic paper preprocessing module, a double-layerquotation network establishing module, a user interesting modelestablishing module and an individualized academic paper recommendingmodule as well as a database. By adopting the recommending system andrecommending method, not only can the correlation of the study contentamong users be used for establishing an author community through asubjective model, but also multiple attribute values of theto-be-recommended author and academic paper inside the community can becalculated, and the weakness that the calculation of the existingrecommending algorithm is large can be improved; and meanwhile, multipleattribute values of the author and academic paper can be simultaneouslycalculated, so that the recommend result is more diversified, and theuser requirement can be better met.

SUMMARY

It is an object of the present technology to ameliorate at least some ofthe inconveniences present in the prior art. Embodiments of the presenttechnology may provide and/or broaden the scope of approaches to and/ormethods of achieving the aims and objects of the present technology.

Developers of the non-limiting embodiments of the present technologyhave observed that, generally speaking, recommendation systems providepersonalized content to users based on previous user interactions withthe recommendation service that can be indicative of user preferencesfor some particular content rather than other content. Typically, therecommended content can come from two principal sources—a native source(native content) and an external source (non-native content).

The external sources are web sites on the Internet, such as newsagencies, news aggregators and other source of content items, which canbe presented to the users of the recommendation systems. On the otherhand, the native sources are “bloggers” that post content using therecommendation system as a platform.

The non-limiting embodiments of the present technology have beendeveloped based on developers' appreciation of at least one problemassociated with the prior art approaches to implementations of therecommendations systems for recommending personalized content to usersand, more specifically but not being so limited, recommendation ofspecialized content.

Within the non-limiting embodiments of the present technology, thespecialized content means content of targeted or “niche” interest.Broadly speaking, some content may be of interest to a large proportionof general population of subscribers or users or the recommendationsystem. An example of such ubiquitously relevant content is an articleabout current affairs. Such an article is likely to be of interest tothe majority of the users of the recommendation system. As anotherexample, an article about an international sports event may of interestto a large portion of the users of the recommendation system, albeit thesize of the pool of users interested in the sports related article maybe smaller than that interested in the current affairs related article.

On another end of the “interest spectrum”, there could be an articleabout a very niche (i.e. specialized) topic. For example, the could becontent item generated by an author of digital content within therecommendation system (i.e. system native content) directed to antiquephoto cameras. Such a digital content item is of interest to a verysmall proportion of the pool of users of the recommendation system. Bothin terms of absolute numbers and comparatively speaking when compared tothe ubiquitously relevant content items (current affairs articles,sports articles, and the like).

As has been alluded to above, in order to recommend a given content itemto the users, the recommendation system needs to predict relevance ofthe given content to a given user. In order to make such a prediction,the recommendation system uses previous interactions of users with thegiven content (amongst other features considered in the relevancyprediction). In other words, when users interact with or “explore” thegiven content item, the recommendation system uses these interactionsfor predicting the relevancy of the given content for recommending thegiven content item to the users (either the same user who performed thepast interactions or a similar user who has a similar user profile or a“lookalike” for short).

However, for those content items available that are of niche interestand therefore are considered to be of “narrow relevancy”—i.e. it ishighly relevant to a smaller portion of the users, while it may becompletely irrelevant to the larger pool of users of the recommendationsystem, the general content selection algorithm of the recommendationsystem may not be appropriate as it will always leave them behind themore generally relevant items.

In other words, some content items (such as, the specialized contentitems) are not often selected by the pool of users (as a whole) or, inother words, not explored often by the general pool of users. Therefore,it is common for this type of content not to be often selected by alarge majority of users and for that reason the amount of userinteractions with such content is very limited. As such, therecommendation system lacks information related to user interactionswith the specialized content.

This results in a cyclical “lack of exploration” problem forrecommendation systems: (i) the recommendation system is not able topredict the relevance of specialized content due to the lack of (or nothaving enough of) previous user interactions therewith; (ii) therecommendation system does not select the specialized content forpotential recommendation to users because the recommendation system isnot able to predict its relevance; and (iii) the users of therecommendation system do not interact with the specialized contentbecause it is not recommended to the users by the recommendation system.

In addition, providing the specialized content to random users does notsolve the above-mentioned cyclical exploration problem for at least tworeasons. Firstly, the specialized content is interesting (relevant) onlyto a narrow pool of users and, therefore, the chances are that randomusers will still ignore the specialized content and will not interacttherewith. Secondly providing the specialized content to random usersdeteriorates the quality of the recommendation system as perceived bythe random users since most of them are not interested in this content.

Therefore, the technical problem that the developers were set out toaddress was how to boost exploration (user interactions) of specializedcontent without deteriorating the quality of the recommendation systemas perceived by the broader pool of users of the recommendation system.

Broadly speaking, the above technical problem is addressed as follows inaccordance with at least of the non-limiting embodiments of the presenttechnology.

As has been alluded to above, some authors of native content of therecommendation systems publish a certain type of specialized content.For example, a first author may publish content related to quantummechanics. In another example, a second author may publish contentrelated to advanced bio-informatics. A given user of the recommendationsystem who is interested in a given type of specialized content maysubscribe to the author who publishes the given type of specializedcontent of interest to the given user, while users who are notinterested in the given type of specialized content do not typicallysubscribe to the author. How the subscription to the channel isimplemented is not particularly limited and may take several forms.

The non-limiting embodiments of the present technology are based on apremise that the subscribers to the given author can be considered as“core users” of the digital content items generated by the given contentauthor/content source. The non-limiting embodiments of the presenttechnology utilize the author-subscriber associations of such core usersfor “boosting” exploration of specialized content originated by suchcontent authors without deteriorating the overall “quality” of therecommendation system as perceived by the general pool users. In otherwords, the non-limiting embodiments of the present technology aim atfinding a proper exploration—exploitation balance in terms of the“one-arm bandit problem”.

In accordance with the non-limiting embodiments of the presenttechnology, the recommendation system is configured to providespecialized content to either just the subscribers of the contentchannel or additionally/alternatively to the users that are similar tothe subscribers (i.e. the pool of core users) of the author thatpublishes the specialized content.

Since the specialized content is provided to users that are similar tothe core users associated with the author that publishes the specializedcontent, the chances are that these users are also interested (as arethe subscribers in the pool of core users) in the specialized contentand are, therefore, more likely to interact therewith which results inan exploration boost of the specialized content. Additionally, userinteraction of the core pool of users enables the recommendation systemto create a base line for the specialized content to be used for rankingspecialized content for showing to all users of the recommendationsystem.

Let it be assumed that a given specialized content author may generatepublished specialized content. As previously alluded to, some users, ifthey like the author's published specialized content, may subscribe to(or otherwise indicate their affiliation with) this author.

As such, these core users are provided by the recommendation system withspecialized content published by the same author in question or similar(thematically) content to that published by the other authors. Now, theinteractions of the core users with such specialized content may be usedas an indication of relevancy of such specialized content not only tothe core user in question, but to the larger population of the users ofthe recommendation system.

Indeed, interactions of the core users indicative of the affiliation tothe new specialized content is an explicit indication by these coreusers that the new specialized content is relevant (interesting) to themand, therefore, may be potentially relevant to users at large. On theother hand, if these specially selected core users are not interested inthis new specialized content—this is a clear indication of the contentvery unlikely to be relevant to the larger pool of users. In otherwords, non-limiting embodiments of the present technology contemplatedrawing conclusions from the subscribers-authors associations in orderto estimate relevancy of the author's specialized content for some otherusers of the general pool of users of the recommendation system.

More specifically, authors generate specialized content and provide iton their publication channels with the recommendation channels. Aspreviously mentioned, users of the system who find the publishedspecialized content of a given author relevant will most likelysubscribe to the publication channel in order to be provided with thenewly published specialized content when it is generated by thespecialized content author, thus, indicating their association to thechannel and the author of the content published on the channel.

Therefore, the recommendation system can identify an initial pool ofsubscribed users for a given specialized content author based on thesubscriptions (or other explicit/implicit association to thechannel/author). Optionally, similar users may be identified in manydifferent ways. For example, “look-alike” features or metrics may beused for identifying similar users. In one implementation, a featurevector is generated for a potentially-similar user. Also, featurevectors for respective subscribed users may be generated. It iscontemplated that an average feature vector may be generated for thepool of subscribed users which, in a sense represents features of anaveraged subscribed user to the author. Then, different algorithms maybe used for determining the distance between this average feature vectorand the feature vector of the potentially-similar user. In somenon-limiting embodiments of the present technology, these similar usersare also added to the pool of core users.

In a sense, the non-limiting embodiments of the present technology“evens outs” the playing field for specialized content with the generalinterest content. It is noted that if the interactions of the core userswith the specialized content bias the ranking of the given content itemupwardly (i.e. if the recommendation system “gets it wrong”) and thecontent is in fact not of interest to the general public, the standardranking MLA used by the recommendation system to select digital contentitems for recommendation will eventually demote the specialized contentand stop showing it.

In accordance with a first broad aspect of the present technology, thereis provided a method of determining a relevancy parameter for a digitalcontent item, the digital content item being originated from a contentchannel associated with a recommendation system. The relevancy parameterfor ranking the digital content item as a recommended content item forusers of the recommendation system, the recommendation system includinga server and at least one client device connectable to the server via acommunication network. The method is executable by the server, theserver further being configured to execute a recommendation algorithm togenerate a set of recommended content items for a given user of therecommendation system. The method comprises: identifying a pool of usersassociated with the content channel, a given user of the pool of usersbeing associated with the content channel; in response to receiving acontent recommendation request from a first client device associatedwith a first user that belongs to the pool of users: generating, usingthe recommendation algorithm, the set of recommended content items forthe first user, a given item of the set of recommended content items notoriginating from the content channel; artificially inserting into theset of recommended items the digital content item; gathering anindication of user interactions of the first user with the set ofrecommended items, the user interactions indicative of the first user'spropensity for the digital content item; and predicting relevancyparameter of the digital content item for a user outside of the pool ofusers based on the user interactions of the first user.

In some implementations of the method, the method further comprises: inresponse to receiving a content recommendation requests from otherclient devices associated with other users that belong to the pool ofusers: generating, using the recommendation algorithm, a respective setof recommended items for the other users, a given item of the respectiveset of recommended items not originating from the content channel;artificially inserting into the respective set of recommended items thedigital content item.

In some implementations of the method, the method further comprisesobserving user interactions of the other users with the respective setsof content recommendations and generating an augmented relevancyparameter associated with the digital content item based on the userinteractions of the other users with the respective sets of contentrecommendations.

In some implementations of the method, the predicting relevancyparameter of the digital content item for the user outside of the poolof users comprises: predicting the relevancy parameter based at least inpart on the augmented relevancy parameter.

In some implementations of the method, the augmented relevancy parameteris upwardly biased relative to a native relevancy parameter that wouldbe generated by the recommendation algorithm.

In some implementations of the method, the method further comprises: inresponse to receiving a content recommendation request from a secondclient device associated with a second user that is outside the pool ofusers: generating, using the recommendation algorithm, the set ofrecommended items for the second user, the set of recommended itemsincluding the digital content item, inclusion of the digital contentitem into the set of recommended items being based on the relevancyparameter; gathering an indication of user interactions of the seconduser with the set of recommended items, the user interactions indicativeof the second user's propensity for the digital content item.

In some implementations of the method, in response to the userinteractions of the second user being indicative of lower propensity forthe digital content item of the second user when compared to the firstuser: adjusting the relevancy parameter of the digital content item to alower value thereof.

In some implementations of the method, the artificially inserting thedigital content item into the set of recommended items comprises:ranking the digital content item relative to other content items withinthe set of recommended items, the ranking being based on the predictedrelevancy parameter of the digital content item.

In some implementations of the method, the artificially inserting thedigital content item into the set of recommended items comprises:positioning the digital content item relative to other content itemswithin the set of recommended items at a pre-determined position withinthe set of recommended items.

In some implementations of the method, the pre-determined position isselected such that to maximize a probability of the user interactionwith the digital content item.

In some implementations of the method, the identifying the pool of usersassociated with the content channel comprises identifying the given userof the pool of users as being associated with the content channel basedon at least one of: (i) an implicit association; and (ii) an explicitassociation.

In some implementations of the method, the implicit associationcomprises the first user having been presented with a prior content itemfrom the content channel and the first user not providing an indicationof negative propensity in response thereto.

In some implementations of the method, the explicit associationcomprises at least one of: the first user subscribing to the contentchannel, the user liking a prior content item from the content channel,and the user commenting on the prior content item from the contentchannel.

In some implementations of the method, the given item of the set ofrecommended items is originating from a network resource accessible viathe communication network.

In some implementations of the method, the given item of the set ofrecommended items is one of a news article, an image, a video, and aninteractive snippet.

In some implementations of the method, all items of the set ofrecommended items are not originating from the content channel.

In some implementations of the method, another given item of the set ofrecommended items is originating from a content channel being one of thecontent channel and another content channel.

In some implementations of the method, the content channel is a nativechannel to the recommendation system.

In accordance with another broad aspect of the present technology, thereis provided a recommendation server for generating a digital contentrecommendation, the digital content recommendation for displaying on anelectronic device associated with a user, the server connectable to theelectronic device via a communication network, the recommendation serverexecuting a ranking algorithm. The recommendation server comprises aprocessor configured to determine a relevancy parameter for a contentitem, the digital content item being originated from a content channelassociated with the recommendation server. The relevancy parameter isfor ranking the digital content item as a recommended content item forusers of the recommendation server, the recommendation server includinga server and at least one client device connectable to the server via acommunication network. The method executable by the recommendationserver, the processor of the recommendation server further beingconfigured to execute a recommendation algorithm to generate a set ofrecommended content items for a given user of the recommendation server.The processor being further configured to: identify a pool of usersassociated with the content channel, a given user of the pool of usersbeing associated with the content channel; in response to receiving acontent recommendation request from a first client device associatedwith a first user that belongs to the pool of users: generate, using therecommendation algorithm, the set of recommended content items for thefirst user, a given item of the set of recommended content items notoriginating from the content channel; artificially insert into the setof recommended items the digital content item; gather an indication ofuser interactions of the first user with the set of recommended items,the user interactions indicative of the first user's propensity for thedigital content item; and predict a relevancy parameter of the digitalcontent item for a user outside of the pool of users based on the userinteractions of the first user.

In the context of the present specification, a “server” is a computerprogram that is running on appropriate hardware and is capable ofreceiving requests (e.g., from client devices) over a network, andcarrying out those requests, or causing those requests to be carriedout. The hardware may be one physical computer or one physical computersystem, but neither is required to be the case with respect to thepresent technology. In the present context, the use of the expression a“server” is not intended to mean that every task (e.g., receivedinstructions or requests) or any particular task will have beenreceived, carried out, or caused to be carried out, by the same server(i.e., the same software and/or hardware); it is intended to mean thatany number of software elements or hardware devices may be involved inreceiving/sending, carrying out or causing to be carried out any task orrequest, or the consequences of any task or request; and all of thissoftware and hardware may be one server or multiple servers, both ofwhich are included within the expression “at least one server”.

In the context of the present specification, “client device” is anycomputer hardware that is capable of running software appropriate to therelevant task at hand. Thus, some (non-limiting) examples of clientdevices include personal computers (desktops, laptops, netbooks, etc.),smartphones, and tablets, as well as network equipment such as routers,switches, and gateways. It should be noted that a device acting as aclient device in the present context is not precluded from acting as aserver to other client devices. The use of the expression “a clientdevice” does not preclude multiple client devices being used inreceiving/sending, carrying out or causing to be carried out any task orrequest, or the consequences of any task or request, or steps of anymethod described herein.

In the context of the present specification, a “database” is anystructured collection of data, irrespective of its particular structure,the database management software, or the computer hardware on which thedata is stored, implemented or otherwise rendered available for use. Adatabase may reside on the same hardware as the process that stores ormakes use of the information stored in the database or it may reside onseparate hardware, such as a dedicated server or plurality of servers.

In the context of the present specification, the expression“information” includes information of any nature or kind whatsoevercapable of being stored in a database. Thus information includes, but isnot limited to audiovisual works (images, movies, sound records,presentations etc.), data (location data, numerical data, etc.), text(opinions, comments, questions, messages, etc.), documents,spreadsheets, lists of words, etc.

In the context of the present specification, the expression “component”is meant to include software (appropriate to a particular hardwarecontext) that is both necessary and sufficient to achieve the specificfunction(s) being referenced.

In the context of the present specification, the expression “computerusable information storage medium” is intended to include media of anynature and kind whatsoever, including RAM, ROM, disks (CD-ROMs, DVDs,floppy disks, hard drivers, etc.), USB keys, solid state-drives, tapedrives, etc.

In the context of the present specification, unless expressly providedotherwise, an “indication” of an information element may be theinformation element itself or a pointer, reference, link, or otherindirect mechanism enabling the recipient of the indication to locate anetwork, memory, database, or other computer-readable medium locationfrom which the information element may be retrieved. For example, anindication of a document could include the document itself (i.e. itscontents), or it could be a unique document descriptor identifying afile with respect to a particular file system, or some other means ofdirecting the recipient of the indication to a network location, memoryaddress, database table, or other location where the file may beaccessed. As one skilled in the art would recognize, the degree ofprecision required in such an indication depends on the extent of anyprior understanding about the interpretation to be given to informationbeing exchanged as between the sender and the recipient of theindication. For example, if it is understood prior to a communicationbetween a sender and a recipient that an indication of an informationelement will take the form of a database key for an entry in aparticular table of a predetermined database containing the informationelement, then the sending of the database key is all that is required toeffectively convey the information element to the recipient, even thoughthe information element itself was not transmitted as between the senderand the recipient of the indication.

In the context of the present specification, the words “first”,“second”, “third”, etc. have been used as adjectives only for thepurpose of allowing for distinction between the nouns that they modifyfrom one another, and not for the purpose of describing any particularrelationship between those nouns. Thus, for example, it should beunderstood that, the use of the terms “first server” and “third server”is not intended to imply any particular order, type, chronology,hierarchy or ranking (for example) of/between the server, nor is theiruse (by itself) intended imply that any “second server” must necessarilyexist in any given situation. Further, as is discussed herein in othercontexts, reference to a “first” element and a “second” element does notpreclude the two elements from being the same actual real-world element.Thus, for example, in some instances, a “first” server and a “second”server may be the same software and/or hardware, in other cases they maybe different software and/or hardware.

Implementations of the present technology each have at least one of theabove-mentioned object and/or aspects, but do not necessarily have allof them. It should be understood that some aspects of the presenttechnology that have resulted from attempting to attain theabove-mentioned object may not satisfy this object and/or may satisfyother objects not specifically recited herein.

Additional and/or alternative features, aspects and advantages ofimplementations of the present technology will become apparent from thefollowing description, the accompanying drawings and the appendedclaims.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the present technology, as well as otheraspects and further features thereof, reference is made to the followingdescription which is to be used in conjunction with the accompanyingdrawings, where:

FIG. 1 depicts a diagram of a system implemented in accordance withnon-limiting embodiments of the present technology.

FIG. 2 depicts a screen shot of a recommendation interface implementedin accordance with a non-limiting embodiment of the present technology,the recommendation interface being depicted as displayed on the screenof an electronic device of the system of FIG. 1, the electronic devicebeing implemented as a smart phone.

FIG. 3 depicts a schematic representation of a process for identifying apool of users associated with a content channel, the process executableby the system of FIG. 1.

FIG. 4 depicts a schematic representation of exploration processexecuted by a content exploration module of the system of FIG. 1.

FIG. 5 depicts a block diagram showing a flow chart of a method, themethod being executable in accordance with the non-limiting embodimentsof the present technology, the method executed by the system of FIG. 1.

DETAILED DESCRIPTION

The examples and conditional language recited herein are principallyintended to aid the reader in understanding the principles of thepresent technology and not to limit its scope to such specificallyrecited examples and conditions. It will be appreciated that thoseskilled in the art may devise various arrangements which, although notexplicitly described or shown herein, nonetheless embody the principlesof the present technology and are included within its spirit and scope.

Furthermore, as an aid to understanding, the following description maydescribe relatively simplified implementations of the presenttechnology. As persons skilled in the art would understand, variousimplementations of the present technology may be of a greatercomplexity.

In some cases, what are believed to be helpful examples of modificationsto the present technology may also be set forth. This is done merely asan aid to understanding, and, again, not to define the scope or setforth the bounds of the present technology. These modifications are notan exhaustive list, and a person skilled in the art may make othermodifications while nonetheless remaining within the scope of thepresent technology. Further, where no examples of modifications havebeen set forth, it should not be interpreted that no modifications arepossible and/or that what is described is the sole manner ofimplementing that element of the present technology.

Moreover, all statements herein reciting principles, aspects, andimplementations of the present technology, as well as specific examplesthereof, are intended to encompass both structural and functionalequivalents thereof, whether they are currently known or developed inthe future. Thus, for example, it will be appreciated by those skilledin the art that any block diagrams herein represent conceptual views ofillustrative circuitry embodying the principles of the presenttechnology. Similarly, it will be appreciated that any flowcharts, flowdiagrams, state transition diagrams, pseudo-code, and the like representvarious processes which may be substantially represented incomputer-readable media and so executed by a computer or processor,whether or not such computer or processor is explicitly shown. Thefunctions of the various elements shown in the figures, including anyfunctional block labeled as a “processor” or a “graphics processingunit”, may be provided through the use of dedicated hardware as well ashardware capable of executing software in association with appropriatesoftware. When provided by a processor, the functions may be provided bya single dedicated processor, by a single shared processor, or by aplurality of individual processors, some of which may be shared. In someembodiments of the present technology, the processor may be a generalpurpose processor, such as a central processing unit (CPU) or aprocessor dedicated to a specific purpose, such as a graphics processingunit (GPU). Moreover, explicit use of the term “processor” or“controller” should not be construed to refer exclusively to hardwarecapable of executing software, and may implicitly include, withoutlimitation, digital signal processor (DSP) hardware, network processor,application specific integrated circuit (ASIC), field programmable gatearray (FPGA), read-only memory (ROM) for storing software, random accessmemory (RAM), and non-volatile storage. Other hardware, conventionaland/or custom, may also be included.

Software modules, or simply modules which are implied to be software,may be represented herein as any combination of flowchart elements orother elements indicating performance of process steps and/or textualdescription. Such modules may be executed by hardware that is expresslyor implicitly shown.

With these fundamentals in place, we will now consider some non-limitingexamples to illustrate various implementations of aspects of the presenttechnology.

Referring to FIG. 1, there is shown a schematic diagram of a system 100,the system 100 being suitable for implementing non-limiting embodimentsof the present technology. It is to be expressly understood that thesystem 100 as depicted is merely an illustrative implementation of thepresent technology. Thus, the description thereof that follows isintended to be only a description of illustrative examples of thepresent technology. This description is not intended to define the scopeor set forth the bounds of the present technology. In some cases, whatare believed to be helpful examples of modifications to the system 100may also be set forth below. This is done merely as an aid tounderstanding, and, again, not to define the scope or set forth thebounds of the present technology. These modifications are not anexhaustive list, and, as a person skilled in the art would understand,other modifications are likely possible. Further, where this has notbeen done (i.e., where no examples of modifications have been setforth), it should not be interpreted that no modifications are possibleand/or that what is described is the sole manner of implementing thatelement of the present technology. As a person skilled in the art wouldunderstand, this is likely not the case. In addition it is to beunderstood that the system 100 may provide in certain instances simpleimplementations of the present technology, and that where such is thecase they have been presented in this manner as an aid to understanding.As persons skilled in the art would understand, various implementationsof the present technology may be of a greater complexity.

Generally speaking, the system 100 is configured to provide contentrecommendations to a user 102 of the system 100. The user 102 may be asubscriber to a recommendation service provided by the system 100.However, the subscription does not need to be explicit or paid for. Forexample, the user 102 can become a subscriber by virtue of downloading arecommendation application from the system 100, by registering andprovisioning a log-in/password combination, by registering andprovisioning user preferences and the like. As such, any systemvariation configured to generate content recommendations for the givenuser can be adapted to execute embodiments of the present technology,once teachings presented herein are appreciated. Furthermore, the system100 will be described using an example of the system 100 being arecommendation system (therefore, the system 100 can be referred toherein below as a “recommendation system 100” or a “prediction system100”). However, embodiments of the present technology can be equallyapplied to other types of the systems 100, as will be described ingreater detail herein below.

The system 100 comprises an electronic device 104, the electronic device104 being associated with the user 102. As such, the electronic device104 can sometimes be referred to as a “client device”, “end user device”or “client electronic device”. It should be noted that the fact that theelectronic device 104 is associated with the user 102 does not need tosuggest or imply any mode of operation—such as a need to log in, a needto be registered, or the like.

The implementation of the electronic device 104 is not particularlylimited, but as an example, the electronic device 104 may be implementedas a personal computer (desktops, laptops, netbooks, etc.), a wirelesscommunication device (such as a smartphone, a cell phone, a tablet andthe like), as well as network equipment (such as routers, switches, andgateways). The electronic device 104 comprises hardware and/or softwareand/or firmware (or a combination thereof), as is known in the art, toexecute a recommendation application 106. Generally speaking, thepurpose of the recommendation application 106 is to enable the user toreceive (or otherwise access) content recommendations provided by thesystem 100, as will be described in greater detail herein below.

How the recommendation application 106 is implemented is notparticularly limited. One example of the recommendation application 106may include a user accessing a web site associated with a recommendationservice to access the recommendation application 106. For example, therecommendation application 106 can be accessed by typing in (orotherwise copy-pasting or selecting a link) an URL associated with therecommendation service. Alternatively, the recommendation application106 can be an app downloaded from a so-called app store, such asAPPSTORE™ or GOOGLEPLAY™ and installed/executed on the electronic device104. It should be expressly understood that the recommendationapplication 106 can be accessed using any other suitable means. In yetadditional embodiments, the recommendation application 106 functionalitycan be incorporated into another application, such as a browserapplication (not depicted) or the like. For example, the recommendationapplication 106 can be executed as part of the browser application, forexample, when the user 102 first start the browser application, thefunctionality of the recommendation application 106 can be executed.

Generally speaking, the recommendation application 106 comprises arecommendation interface 108, the recommendation interface 108 beingdisplayed on a screen (not separately numbered) of the electronic device104. With reference to FIG. 2, there is depicted a screen shot of therecommendation interface 108 implemented in accordance with anon-limiting embodiment of the present technology (the example of therecommendation interface 108 being depicted as displayed on the screenof the electronic device 104 being implemented as a smart phone).

In some embodiments of the present technology the recommendationinterface 108 is presented when the user 102 of the electronic device104 actuates (i.e. executes, run, background-run or the like) therecommendation application 106. Alternatively, the recommendationinterface 108 can be presented when the user 102 opens a new browserwindow and/or activates a new tab in the browser application. Forexample, in some embodiments of the present technology, therecommendation interface 108 can act as a “home screen” in the browserapplication.

The recommendation interface 108 includes a search interface 202. Thesearch interface 202 includes a search query interface 204. The searchquery interface 204 can be implemented as an “omnibox” which allowsentry of a search query for executing a search or a network address(such as a Universal Remote Locator) for identifying a network resource(such as a web site) to be accessed. However, the search query interface204 can be implemented as configured to receive one or both of: entry ofthe search query for executing the search or the network address (suchas a Universal Remote Locator) for identifying the network resource(such as a web site) to be accessed.

The recommendation interface 108 further includes a links interface 206.The links interface 206 includes a plurality of tiles 208—of which eightare depicted in FIG. 2 - only two of which are numbered in FIG. 2—afirst tile 210 and a second tile 212.

Using the example of the first tile 210 and the second tile 212—each ofthe plurality of tiles 208 includes (or acts as) a link to either (i) aweb site marked as “favourite” or otherwise marked by the user 102, (ii)a previously visited web site or (iii) the like. The plurality of tiles208, in the depicted embodiment, is visually presented to the user 102as square buttons with a logo and/or a name of the resource depictedtherein, the logo and the name for enabling the user 102 to identifywhich resource the particular one of the plurality of tiles (notseparately numbered) is linked to. However, it should be expresslyunderstood that the visual representation of some or all of theplurality of tiles 208 can be different. As such, some or all of theplurality of tiles 208 can be implemented as differently shaped buttons,as hyperlinks presented in a list or the like.

As an example, the first tile 210 contains a link to a TRAVELZOO™ website and the second tile 212 contains a link to a personal live journalweb site. Needless to say, the number and content of the individual onesof the plurality of tiles 208 is not particularly limited.

For example, the number of the tiles within the plurality of tiles 208can be pre-selected by the provider of the recommendation application106. In some embodiments of the present technology, the number of tileswithin the plurality of tiles 208 is pre-selected based on the sizeand/or resolution of the screen of the electronic device 104 executingthe recommendation application 106. For example, a first number of tilescan be pre-selected for the electronic device 104 executed as asmartphone, a second number of tiles can be pre-selected for theelectronic device 104 executed as a tablet, and a third number of tilescan be pre-selected for the electronic device 104 executed as a laptopor desktop computer.

The recommendation interface 108 further includes a recommended contentset 214. The recommended content set 214 includes one or morerecommended content items, such as a first recommended content item 216and a second recommended content item 218 (the second recommendedcontent item 218 only partially visible in FIG. 2). Naturally, therecommended content set 214 can have more recommended content items. Inthe embodiment depicted in FIG. 2 and in those embodiments where morethan one recommended content item are present, the user 102 can scrollthrough the recommended content set 214. The scrolling can be achievedby any suitable means. For example, the user 102 can scroll the contentof the recommended content set 214 by means of actuating a mouse device(not depicted), a key board key (not depicted) or interacting with atouch sensitive screen (not depicted) of or associated with theelectronic device 104.

Example provided in FIG. 2 is just one possible implementation of therecommendation interface 108. Another example of the implementation ofthe recommendation interface 108, as well as an explanation of how theuser 102 can interact with the recommendation interface 108 is disclosedin a co-owned United States Patent Application entitled ACOMPUTER-IMPLEMENTED METHOD OF GENERATING A CONTENT RECOMMENDATIONINTERFACE, filed on May 11, 2017 and bearing a publication number2017032949 0 Al; content of which is incorporated by reference herein inits entirety.

How the content for the recommended content set 214 is generated will bedescribed in greater detail herein below.

Returning to the description of FIG. 1, the electronic device 104 iscommunicatively coupled to a communication network 110 for accessing arecommendation server 112. In some non-limiting embodiments of thepresent technology, the communication network 110 can be implemented asthe Internet. In other embodiments of the present technology, thecommunication network 110 can be implemented differently, such as anywide-area communication network, local-area communication network, aprivate communication network and the like. A communication link (notseparately numbered) between the electronic device 104 and thecommunication network 110 is implemented will depend inter alia on howthe electronic device 104 is implemented. Merely as an example and notas a limitation, in those embodiments of the present technology wherethe electronic device 104 is implemented as a wireless communicationdevice (such as a smartphone), the communication link can be implementedas a wireless communication link (such as but not limited to, a 3Gcommunication network link, a 4G communication network link, WirelessFidelity, or WiFi® for short, Bluetooth® and the like). In thoseexamples where the electronic device 104 is implemented as a notebookcomputer, the communication link can be either wireless (such asWireless Fidelity, or WiFi® for short, Bluetooth® or the like) or wired(such as an Ethernet based connection).

The recommendation server 112 can be implemented as a conventionalcomputer server. In an example of an embodiment of the presenttechnology, the recommendation server 112 can be implemented as a Dell™PowerEdge∩ Server running the Microsoft™ Windows Server™ operatingsystem. Needless to say, the recommendation server 112 can beimplemented in any other suitable hardware, software, and/or firmware,or a combination thereof. In the depicted non-limiting embodiments ofthe present technology, the recommendation server 112 is a singleserver. In alternative non-limiting embodiments of the presenttechnology, the functionality of the recommendation server 112 may bedistributed and may be implemented via multiple servers.

The recommendation server 112 comprises a processing module 114. Theprocessing module 114 is coupled to or otherwise has access to, acontent discovery module 115, an analytics module 116, and a recommendedcontent selection module 117. The recommendation server 112 has accessto a data storage device 118. Operation of the recommendation server 112and its components will be described herein below in greater detail.

The data storage device 118 includes a main database 120, an itemfeature database 122, a recommendable non-native content item database124, a recommendable native content item database 125, and a userinteraction database 126.

Also coupled to the communication network 110 are a plurality of networkresources 130, including a first network resource 132, a second networkresource 134 and a plurality of additional network resources 136. Thefirst network resource 132, the second network resource 134 and theplurality of additional network resources 136 are all network resourcesaccessible by the electronic device 104 (as well as other electronicdevices potentially present in the system 100) via the communicationnetwork 110. In accordance with the non-limiting embodiments of thepresent technology, the first network resource 132, the second networkresource 134 and the plurality of additional network resources 136 canbe considered “non-native sources” by virtue of them being potentialsources of non-native digital content items that can be recommended bythe recommendation server 112.

The type of the respective content of first network resource 132, thesecond network resource 134 and the plurality of additional networkresources 136 is not particularly limited.

A given one of the first network resource 132, the second networkresource 134 and the plurality of additional network resources 136 cancontain (or in other words, host) digital content (i.e. one or moredigital items each of the one or more digital items having one or moretypes of digital content). In some embodiments of the presenttechnology, the content of the digital items can include but is notlimited to: audio content for streaming or downloading, video contentfor streaming or downloading, news, blogs, information about variousgovernment institutions, information about points of interest,thematically clustered content (such as content relevant to thoseinterested in kick-boxing), other multi-media content, and the like.

In other embodiments of the present technology, the content of thedigital items hosted by the first network resource 132, the secondnetwork resource 134 and the plurality of additional network resources136 is text-based. Examples of the text-based content items include butare not limited to: news, articles, blogs, information about variousgovernment institutions, information about points of interest,thematically clustered content (such as content relevant to thoseinterested in kick-boxing), and the like. It should be noted that“text-based” content does not intend to mean that the given digital itemonly contains text to the exclusion of other type of multi-mediaelements. On the contrary, the given text-based digital item includestext elements, as well as potentially other type of multi-mediaelements. For instance, a given text-based digital item that is anarticle may have text, as well as photos. As another example, a giventext-based digital item that is a blog may include text, as well asembedded video elements.

The content is potentially “discoverable” to the electronic device 104by various means.

For example, the user 102 of the electronic device 104 can use a browserapplication (not depicted) and enter a Universal Resource Locator (URL)associated with the given one of the first network resource 132, thesecond network resource 134 and the plurality of additional networkresources 136. Alternatively, the user 102 of the electronic device 104can execute a search using a search engine (not depicted) to discoverthe content of one or more of the first network resource 132, the secondnetwork resource 134 and the plurality of additional network resources136. As has been mentioned above, these are useful when the user 102knows apriori which content the user 102 is interested in.

As has been alluded to above, in some embodiments of the presenttechnology, the recommendation server 112 may provide a platform fordigital content generation and publication. This can be particularlyconvenient for those users of the recommendation server 112 who wish topublish digital content but do not wish to spend time and/or significantamount of money for establishing a publication platform. It is notedthat the publication platform provided by the recommendation server 112can be provided based on a subscription, in exchange for the subscribershaving to watch ads and/or for free and/or for a fee. Just as anexample, the first native content channel may be associated with ablogger that publishes content items (such as the first content item122) using the recommendation application 106 as a platform. Therecommendable native content item database 125 can store native digitalcontent items. For example, a given content item stored in therecommendable native content item database 125 can originate from anative content channel of the recommendation server 112.

In accordance with the non-limiting embodiments of the presenttechnology, the recommendation application 106 can recommend contentitems that the user 102 may not apriori know about. These recommendedcontent items can originate from either (i) the given one of the firstnetwork resource 132, the second network resource 134 and the pluralityof additional network resources 136 to the user 102; or (ii) one or morenative content channels associated with the recommendation server 112.

The recommendation server 112 is configured to select content for theone or more recommendation items to be presented to the user 102 via therecommendation application 106. More specifically, the processing module114 is configured to (i) receive from the electronic device 104 arequest for the content recommendation 150 and (ii) responsive to therequest, generate a recommended content message 152 specificallycustomized for the user 102 associated with the electronic device 104.The processing module 114 can further coordinate execution of variousroutines described herein as performed by the content discovery module115, the analytics module 116, and the recommended content selectionmodule 117 for example.

The processing module 114 is configured to store, in the main database120, information extracted during processing. Generally speaking, themain database 120 may receive data from the processing module 114 thatwas extracted or otherwise determined by the processing module 114during processing for temporary and/or permanent storage thereof and mayprovide stored data to the processing module 114 for use thereof.

In some embodiments of the present technology, the request for thecontent recommendation 150 can be generated in response to the user 102providing an explicit indication of the user desire to receive thecontent recommendation. For example, the recommendation interface 108can provide a button (or another actuatable element) to enable the user102 to indicate her/his desire to receive a new or an updated contentrecommendation. As a non-limiting example, the recommendation interface108 can provide an actuatable button that reads “Request a contentrecommendation”. Within these embodiments, the request for the contentrecommendation 150 can be thought of as “an explicit request” in a senseof the user 102 expressly providing a request for the recommendedcontent.

In other embodiments, the request for the content recommendation 150 canbe generated in response to the user 102 providing an implicitindication of the user desire to receive the content recommendation. Insome embodiments of the present technology, the request for the contentrecommendation 150 can be generated in response to the user 102 startingthe recommendation application 106.

Alternatively, in those embodiments of the present technology where therecommendation application 106 is implemented as a browser (for example,a GOOGLE™ browser, a YANDEX™ browser, a YAHOO! ™ browser or any otherproprietary or commercially available browser application), the requestfor content recommendation 150 can be generated in response to the user102 opening the browser application and can be generated, for example,without the user 102 executing any additional actions other thanactivating the browser application. As another example, the request forcontent recommendation 150 can be generated in response to the user 102opening a new tab of the already-opened browser application and can begenerated, for example, without the user 102 executing any additionalactions other than activating the new browser tab. In other words, therequest for the content recommendation 150 can be generated even withoutthe user 102 knowing that the user 102 may be interested in obtaining acontent recommendation.

As another example, the request for content recommendation 150 can begenerated in response to the user 102 selecting a particular element ofthe browser application and can be generated, for example, without theuser 102 executing any additional actions other thanselecting/activating the particular element of the browser application.

Examples of the particular element of the browser application includebut are not limited to:

-   -   An address line of the browser application bar    -   A search bar of the browser application and/or a search bar of a        search engine web site accessed in the browser application    -   An omnibox (combined address and search bar of the browser        application)    -   A favourites or recently visited network resources pane    -   Any other pre-determined area of the browser application        interface or a network resource displayed in the browser        application

How the recommended content selection module 117 recommends contentitems in response to the request for the content recommendation 150 willbe explained in more detail herein below.

In some embodiments of the present technology, the content discoverymodule 115 can be configured to execute a “crawler” operation. In otherwords, the content discovery module 115 can execute a robot that“visits” a plurality of resources (such as the plurality of networkresources 130 including the first network resource 132, the secondnetwork resource 134 and the plurality of additional network resources136) and catalogues one or more digital items hosted by a respective oneof the first network resource 132, the second network resource 134 andthe plurality of additional network resources 136. In some embodimentsof the present technology, the content discovery module 115 cancatalogue the digital items into an inverted index mapping a givendigital item to a list of key words associated with the given digitalitem.

As part of the crawling function, the content discovery module 115 isconfigured to maintain information representative of the newlydiscovered network resources and/or newly discovered content availabletherefrom. In some embodiments, the content discovery module 115 can beconfigured to maintain an inverted index as an example in therecommendable non-native content item database 124 within the datastorage device 118, but the recommended content selection module 117 canarrange the information representative of the newly discovered networkresources and/or newly discovered content available therefrom in a datastructure other than an inverted index. In other embodiments, thecontent discovery module 115 may also be configured to extract featuresfrom the newly discovered network resources and/or newly discoveredcontent available, and store the associated features, as a non-limitingexample, in the item feature database 122.

In alternative embodiments of the present technology, rather thanexecuting its dedicated content discovery module 115, the recommendationserver 112 can share the functionality of content discovery module 115with another server (not depicted) and/or another service (notdepicted). For example, the functionality of the content discoverymodule 115 can be shared with a search engine server (not depicted)executing a search engine service. When the content discovery module 115crawls and indexes new resources that may potentially host text-based orother digital items, the content discovery module 115 can also indexsuch newly discovered (or updated) digital items for the purposes of therecommendation server 112 routines described herein.

The recommendable non-native content item database 124 storesinformation/content associated with a pool of potentially recommendablenon-native content items by the recommendation service and thatcomprises some or all the digital content items discovered by thecontent discovery module 115.

The nature of one or more potentially recommendable non-native contentitems within the pool of recommendable non-native content items is notparticularly limited. Some examples of the one or more potentiallyrecommendable non-native content items include but are not limited todigital content items such as:

-   -   a news item;    -   a publication;    -   a web resource;    -   a post on a social media web site;    -   a new item to be downloaded from an application store;    -   a new song (music track) to play/download from a resource;    -   an audiobook to play/download from a resource;    -   a podcast to play/download from a resource;    -   a new movie (video clip) to play/download from a resource;    -   a product to be bought from a resource; and    -   a new document uploaded for viewing on a social media web site        (such as a new photo uploaded to an INSTRAGRAM™ or FACEBOOK™        account).

By the same token, the recommendable native content item database 125information/content associated with a pool of potentially recommendablenative content items by the recommendation service.

The nature of one or more potentially recommendable native content itemswithin the pool of recommendable native content items is notparticularly limited. Some examples of the one or more potentiallyrecommendable native content items include but are not limited todigital content items such as:

-   -   a user-generated publication;    -   a user-generated blog post;    -   a user-generated photograph;    -   a user-generated video.

Thus, it can be said that the pool of potentially recommendable contentitems can comprise (i) at least one item from the respective pluralitiesof items associated with the plurality of network resources 130 and (ii)at least one content item from the one or more native content sources.The pool of potentially recommendable content items may then berecommended to users of the recommendation service by the recommendedcontent selection module 117.

The analytics module 116 is configured to (i) track interactions ofusers with content items that were previously recommended by therecommendation service, and store the user interactions in the userinteraction database 126, and (ii) extract information related to itemfeatures associated with, for example, content items that werepreviously recommended by the recommendation service to its previoususers and with which at least one previous user has interacted in theitem feature database 122.

Examples of user events/interactions associated with previous users ofthe system 100 tracked by the analytics module 116 and stored in theuser interaction database 126 include but are not limited to:

-   -   a given user of the recommendation system “scrolled over” a        given item;    -   a given user of the recommendation system “liked” or “disliked”        the given item;    -   a given user of the recommendation system shared the given item;    -   a given user of the recommendation system has clicked on (or        otherwise selected) the given item;    -   a given user of the recommendation system has spent time        consulting the given item before returning to the recommendation        system; and    -   a given user of the recommendation system has        purchased/ordered/downloaded the given item.

As a non-limiting example, each of the user events/interactions in theuser interaction database 126 may be associated with a respectivetimestamp, a respective content item, and a respective user.

Examples of item features extracted by the analytics module 116 andstored in the item feature database 122 include but are not limited to:

-   -   popularity of a given item amongst users of the recommendation        service (for example, in case of the given item being a music        track, how many times this given music track has been listened        to/downloaded by users of the recommendation service);    -   a number of likes/purchases/downloads/clicks amongst all events        associated with the given item and that have been performed via        the recommendation service; and    -   item-inherent characteristics that are based on content of the        respective content item—in case of the given item being a music        track—length of the track, the genre of the track,        audio-characteristic of the track (for example, tempo of the        track); other item-inherent characteristics include: the price        of the given item, the dimensions of the given item, the        category of the given item, the producer/maker of the item, the        length of the document measured in words or symbols;        category/theme of the document; movie rating on a movie rating        host, etc.

It should be expressly understood that the user events and the itemfeatures may take many forms and are not specifically limited. As such,above presented lists of non-limiting examples of the way that the userevents and the item features may be implemented are just examplesthereof. As such, it should be expressly understood that many otheralternative implementations of the user events and the item features maybe contemplated in different implementations of the present technology.

How information is obtained and stored in the item feature database 122,the recommendable non-native content item database 124, therecommendable native content item database 125 and the user interactiondatabase 126 is not particularly limited.

For example, the information related to the item features may beobtained from a particular service that maintains information aboutvarious items available therefrom and the like; and stored in the itemfeature database 122. The information related to the item features maybe divided into various categories representative of various types ortopics of items.

The recommended content selection module 117 can be configured toexecute one or more machine learning algorithms (MLAs) to recommendcontent items to users of the content recommendation service. In someembodiments of the present technology, one or more machine learningalgorithms can be any suitable or semi-supervised supervised machinelearning algorithm, such as but not limited to:

-   -   Artificial neural network    -   Bayesian statistics    -   Gaussian process regression    -   Decision trees    -   And the like.

Generally speaking, the recommended content selection module 117executes one or more MLAs to analyze the indexed content items (i.e.those discovered and indexed by the content discovery module 115 in therecommendable non-native content item database 124 and the recommendablenative content item database 125 within the data storage device 118) toselect one or more of the digital content items as recommended contentitems for the user 102. The one or more MLAs executed by the recommendedcontent selection module 117, may as an example, in response to therequest for the content recommendation 150, select one or more indexedcontent items as recommended content items for the user 102 based on (i)one or more item features of the indexed content items from the itemfeature database 122; and (ii) previous user interactions with indexedcontent items (associated with the user 102 or other users of therecommendation service) in the user interaction database 126.

It should be noted that even though the content discovery module 115,the analytics module 116, and the recommended content selection module117 have been described as separate entities each executing itsrespective functionalities, in alternative embodiments of the presenttechnology, the functionality executed by the content discovery module115, the analytics module 116, and the recommended content selectionmodule 117 can be executed by a single entity (such as the processingmodule 114, for example). Alternatively, the functionality executed thecontent discovery module 115, the analytics module 116, and therecommended content selection module 117 can be distributed amongst moremodules than the ones depicted in FIG. 1 and can be executed as part ofmultiple instances of the recommendation server 112.

Furthermore, each one of the content discovery module 115, the analyticsmodule 116, and the recommended content selection module 117 can executeadditional functions (i.e. others than the respective functionsdescribed herein).

It should be noted that even though the main database 120, the itemfeature database 122, the recommendable non-native content item database124, recommendable native content item database 125, and the userinteraction database 126 are depicted as separate databases, this doesnot need to be so in each and every embodiment of the presenttechnology. As such, some or all of the main database 120, the itemfeature database 122, the recommendable non-native content item database124, recommendable native content item database 125, and the userinteraction database 126 may be implemented in a single database.Furthermore, any one of the main database 120, the item feature database122, the recommendable non-native content item database 124,recommendable native content item database 125, and the user interactiondatabase 126 may, in itself, be split into several distributed storagesdevices (not depicted).

In accordance with the non-limiting embodiments of the presenttechnology, the recommended content selection module 117 is configured acontent exploration module 140. Broadly speaking, the contentexploration module 140 is configured to determine a relevancy parameterfor a digital content item. In a specific non-limited embodiment of thepresent technology, the content exploration module 140 is configured todetermine the relevancy parameter for one or more native digital contentitems. In a specific non-limited embodiment of the present technology,the content exploration module 140 is configured to determine therelevancy parameter for one or more native digital content items thatare associated with a specialized topic. In other words, the contentexploration module 140 can be configured to determine the relevancyparameter for one or more native digital content items that areassociated with a topic of niche interest. In alternative non-limitingembodiments of the present technology, the content exploration module140 can configured to determine the relevancy parameter for one or morenative digital content items that do not have enough past userinteraction data stored in the user interaction database 126.

In yet further non-limiting embodiments of the present technology, themethods described herein below can be applied to determining therelevancy parameter for one or more non-native digital content itemsthat do not have enough past user interaction data stored in the userinteraction database 126.

In a specific non-limiting embodiment of the present technology, thecontent exploration module 140 is configured to determine the relevancyparameter for one or more native digital content items that areoriginated from a content channel of the system 100. The content channelcan be associated with a channel author, who in turn can be a subscriberto the publication platform provided within the system 100, as has beenalluded to above.

With reference to FIG. 3, there is depicted a schematic representationof a process 300 for identifying a pool of users 306 associated with acontent channel 301. As depicted in FIG. 3, the content channel 301originates at least one digital content item—shown in FIG. 3 as adigital content item 302.

Also depicted in FIG. 3 is a plurality of users 304. The plurality ofusers 304 include all users of the system 100—i.e. those users whoaccess the system 100 for the receiving digital content itemsrecommendations. There is also depicted the pool of users 306, which isa subset of the plurality of users 306. The pool of users 306 containthose users of the plurality of users 304 who are associated with thecontent channel 301. These associations are schematically depicted inFIG. 3 at 308.

A given association 308 can be at least one of: (i) an implicitassociation; and (ii) an explicit association. In some non-limitingembodiments of the present technology, the implicit association canmanifest itself in the user 102 having been presented with a priorcontent item from the content channel 301 and the user 102 not providingan indication of negative propensity in response thereto. On the otherhand, the explicit association can manifest itself in at least one of:the user 102 subscribing to the content channel 301, the user 102 likinga prior content item from the content channel 301, and the user 102commenting on the prior content item from the content channel 301.

As such, in accordance with the non-limiting embodiments of the presenttechnology, the content exploration module 140 determines the pool ofusers 306 can be thought as “core users” for the content channel 301.The content exploration module 140 can repeat such determination for allother content channels similar to the content channel 301. In somenon-limiting embodiments of the present technology, such determinationof the pool of users 306 can be executed “offline”. In accordance withthe non-limiting embodiments of the present technology, the term“offline” refers to a moment in time prior to receiving a digitalcontent recommendation request (i.e. the aforementioned request for thecontent recommendation 150) from the user 102. In some other specificnon-limiting embodiments of the present technology, the term “offline”may additionally or alternatively mean a moment in time, when the demandfor the content recommendations can be low (such as, for example, in amiddle of the night or the like).

With reference to FIG. 4, there is depicted a schematic representationof exploration process 400 executed by the content exploration module140. When the content exploration module 140 receives a given instanceof the content recommendation 150. The content exploration module 140then determines, as part of a step 402 of the exploration process 400,whether or not the content recommendation 150 is originated from theuser 102 who is part of any of the pools of the core users, such forexample, the pool of users 306 associated with the content channel 301.

In response to the content exploration module 140 determining (at step404 of the exploration process 400) that the content recommendation 150has indeed originated from the user 102 who is part of any of the poolsof the core users, such for example, the pool of users 306 associatedwith the content channel 301; the content exploration module 140executes a relevancy parameter exploration routine 405.

In response to the content exploration module 140 determining (at step406 of the exploration process 400) that the content recommendation 150has not originated from the user 102 who is part of any of the pools ofthe core users, such for example, the pool of users 306 associated withthe content channel 301; the content exploration module 140 executes astandard content recommendation routine 407.

In some non-limiting embodiments of the present technology, the contentexploration module 140 executes the relevancy parameter explorationroutine 405 for each one of the content channels 301. In some otherspecific non-limiting embodiments of the present technology, the contentexploration module 140 executes the relevancy parameter explorationroutine 405 for each one of the content channels 301 that is determinedto be associated with specialized content. In some other specificnon-limiting embodiments of the present technology, the contentexploration module 140 executes the relevancy parameter explorationroutine 405 for each one of the content channels 301 that is determinedto be associated with niche content. In some other specific non-limitingembodiments of the present technology, the content exploration module140 executes the relevancy parameter exploration routine 405 for eachone of the content channels 301 that lacks past user interactioninformation (i.e. no data or limited data stored in the user interactiondatabase 126 in associated with the digital content originated from thecontent channel 301).

In some non-limiting embodiments of the present technology, for thegiven one of the content channels 301, the content exploration module140 continues executing the relevancy parameter exploration routine 405until sufficient explorative information is obtained (will be explainedin greater detail herein below). In other non-limiting embodiments ofthe present technology, the content exploration module 140 can repeatexecuting the relevancy parameter exploration routine 405 from time totime, even after the sufficient explorative information is obtained, forexample, in order to update the explorative information, which maychange over time.

As part of the relevancy parameter exploration routine 405, the contentexploration module 140 causes the recommended content selection module117 as part of generating the set of recommended content items for theuser 102 to artificially insert into the set of recommended items anexplorative content item originating from the content channel 301, suchas the digital content item 302.

In some non-limiting embodiments of the present technology, theartificially inserting the digital content item into the set ofrecommended items comprises: ranking the digital content item relativeto other content items within the set of recommended items, the rankingbeing based on the predicted relevancy parameter of the digital contentitem. This can be done, for example, by amending or over-riding theranking algorithm.

In some non-limiting embodiments of the present technology, theartificially inserting the digital content item into the set ofrecommended items comprises: positioning the digital content itemrelative to other content items within the set of recommended items at apre-determined position within the set of recommended items. Forexample, the artificially inserting the digital content item may includepositioning the digital content item on a pre-determined position, suchas a first position, a second position, or the like.

In some non-limiting embodiments of the present technology, thepre-determined position is selected such that to maximize a probabilityof the user interaction with the digital content item. Such position canbe determined empirically or using any known WIN—LOSS algorithm

Broadly speaking, in accordance with the non-limiting embodiments of thepresent technology, the relevancy parameter exploration routine 405 isbuilt on the premise of exploring content relevancy of specializedcontent using core users that are associated with the content channel301 that is at least tangentially related to such specialized content.

Without wishing to be bound to any specific theory, embodiments of thepresent technology are built on the premise that even if the core users(i.e. those in the pool of users 306) who have demonstrated affinity tothe content channel 301 do not show interest in the digital content item302 (through a positive interaction, click or any otheraffinity-demonstrating activity), then the digital content item 302 isunlikely to be of interest to a wide population of users of the system100 and, thus, should be associated with a lower value of the relevancyparameter.

On the other hand, if the core users (i.e. those in the pool of users306) who have demonstrated affinity to the content channel 301 do showinterest in the digital content item 302 (through a positiveinteraction, click or any other affinity-demonstrating activity), thenthe digital content item 302 may potentially be of interest to the widerpopulation of users of the system 100 and, thus, should be associatedwith a lower value of the relevancy parameter. In this case, therelevancy parameter exploration routine 405 can assign the digitalcontent item 302 with a higher value of the relevancy parameter, whichcan then then be used for ranking the digital content item 302 as partof the standard content recommendation routine 407.

Therefore and without wishing to be bound to any specific theory, theembodiments of the present technology and, specifically, execution ofthe relevancy parameter exploration routine 405 allows to “promote” thespecialized/niche content for inclusion in the sets of recommended itemsas part of the standard content recommendation routine 407. Withoutimplementing embodiments of the present technology, suchspecialized/niche content may never become part of the sets ofrecommended items and, therefore, may never “have a chance” to collectenough user interaction history to be fairly judged by the rankingalgorithm of the system 100. Embodiments of the present technology allowfor the specialized/niche content to “start on the same foot” as thegeneral popularity content as far as the ranking algorithm of the system100 is concerned.

It should be noted, however, if it turns out that the given specializedcontent of the digital content item 302 was of interest to the coreusers (and, therefore, got a higher value of the relevancy parameter bythe relevancy parameter exploration routine 405, but indeed was not ofinterest to the general population of the users of the recommendationsystem, the relevancy parameter of the digital content item 302 willeventually get pessimized by lack of interactions of other users whenthe digital content item 302 is recommended to the other users as partof the standard content recommendation routine 407.

In other words, embodiments of the present technology contemplate; (i)gathering an indication of user interactions of the user 102 with theset of recommended items, the user interactions indicative of the firstuser's propensity for the specialized content item (as part of therelevancy parameter exploration routine 405); in order to predictrelevancy parameter of the digital content item 302 for a user outsideof the pool of users 306 based on the user interactions of the user 102in (i) above (the predicting being done as part of the standard contentrecommendation routine 407).

Given the architecture described above, it is possible to execute amethod of determining a relevancy parameter for a digital content item,the digital content item being originated from a content channel 301associated with the recommendation system 100. In accordance with thenon-limiting embodiments of the present technology, the relevancyparameter can be used for ranking the digital content item as arecommended digital content item for users of the recommendation system100 using a recommendation algorithm of the recommendation system 100.

With reference to FIG. 5, there is depicted a block diagram showing aflow chart of a method 500, the method 500 being executable inaccordance with the non-limiting embodiments of the present technology.The method 500 can be executed by the content exploration module 140 ofthe recommendation server 112.

Step 502—identifying a pool of users associated with the contentchannel, a given user of the pool of users being associated with thecontent channel based on at least one of: (i) an implicit association;and (ii) an explicit association

At step 502, the content exploration module 140 of the recommendationserver 112 identifies a pool of users associated with the contentchannel. As has been described above the content exploration module 140identifies the pool of users 306 associated with the content channel301. The given user of the pool of users 306 can be associated with thecontent channel based 301 on at least one of: (i) an implicitassociation; and (ii) an explicit association.

Step 504—in response to receiving a content recommendation request froma first client device associated with a first user that belongs to thepool of users: generating, using the recommendation algorithm, the setof recommended content items for the first user, a given item of the setof recommended content items not originating from the content channel;artificially inserting into the set of recommended items the digitalcontent item; gathering an indication of user interactions of the firstuser with the set of recommended items, the user interactions indicativeof the first user's propensity for the digital content item

At step 504, in response to receiving a content recommendation requestfrom a first client device associated with a first user that belongs tothe pool of users 306, the content exploration module 140 of therecommendation server 112 executes: generating, using the recommendationalgorithm, the set of recommended content items for the first user, agiven item of the set of recommended content items not originating fromthe content channel; artificially inserting into the set of recommendeditems the digital content item; gathering an indication of userinteractions of the first user with the set of recommended items, theuser interactions indicative of the first user's propensity for thedigital content item.

It should be noted that the steps 502 and 504 of the method 500 can bebroadly categorized as a “content relevancy exploration phase”. In otherwords, the steps 502 and 504 are the steps o the method 500 when thecontent exploration module 140 of the recommendation server 112 learnsthe relevancy parameter of the specialized digital content (as anexample) using the pool of users 306.

In some non-limiting embodiments of the present technology, the contentexploration module 140 of the recommendation server 112 can perform thecontent exploration phase off-line relative to actually providing therecommended digital content sets (such as, for example, during off peakhours and the like).

In some non-limiting embodiments of the present technology, the contentexploration module 140 of the recommendation server 112 can perform thecontent exploration phase for each one of the content channels 301 inaccordance to a pre-determined or a randomly generated schedule.

In other non-limiting embodiments of the present technology, the contentexploration module 140 of the recommendation server 112 can perform thecontent exploration phase for a subset of the content channels 301, suchas the top 10%, the top 20% or any other proportion of most popular onesof the content channels 301.

Step 506—predicting relevancy parameter of the digital content item fora user outside of the pool of users based on the user interactions ofthe first user

At step 506, the recommended content selection module 117 predictsrelevancy parameter of the digital content item for a user outside ofthe pool of users 306 based on the user interactions of the first user.

In other words, the step 506 can be thought of as an in-use phase of themethod 500. Where the recommended content selection module 117 uses the“explored and predicted relevancy parameter” to predict relevancy of theassociated digital content item to the general population of the user ofthe recommendation service of the recommendation system 100.

Optional/Alternative Implementations of the Method 500

In some embodiments of the method 500, the method 500 further comprises:in response to receiving a content recommendation requests from otherclient devices associated with other users that belong to the pool ofusers 306: generating, using the recommendation algorithm, a respectiveset of recommended items for the other users, a given item of therespective set of recommended items not originating from the contentchannel 301; artificially inserting into the respective set ofrecommended items the digital content item.

In some non-limiting embodiments of the present technology, the method500 further comprises observing user interactions of the other userswith the respective sets of content recommendations and generating anaugmented relevancy parameter associated with the digital content itembased on the user interactions of the other users with the respectivesets of content recommendations. In some non-limiting embodiments of thepresent technology, the predicting relevancy parameter of the digitalcontent item for the user outside of the pool of users 306 comprises:predicting the relevancy parameter based at least in part on theaugmented relevancy parameter. In some non-limiting embodiments of thepresent technology, the augmented relevancy parameter is upwardly biasedrelative to a native relevancy parameter that would be generated by therecommendation algorithm.

Thus, as has been alluded to above, the exploration process can berepeated with various core users until sufficient prior user interactioninformation is obtained.

In some non-limiting embodiments of the present technology, the method500 further comprises: in response to receiving a content recommendationrequest from a second client device associated with a second user thatis outside the pool of users: generating, using the recommendationalgorithm, the set of recommended items for the second user, the set ofrecommended items including the digital content item, inclusion of thedigital content item into the set of recommended items being based onthe relevancy parameter; gathering an indication of user interactions ofthe second user with the set of recommended items, the user interactionsindicative of the second user's propensity for the digital content item.

In some non-limiting embodiments of the present technology, in responseto the user interactions of the second user being indicative of lowerpropensity for the digital content item of the second user when comparedto the first user: adjusting the relevancy parameter of the digitalcontent item to a lower value thereof. This process has been describedabove as the “pessimization” process of the predicted relevancyparameter.

In some non-limiting embodiments of the present technology, the implicitassociation comprises the first user having been presented with a priorcontent item from the content channel and the first user not providingan indication of negative propensity in response thereto.

In some non-limiting embodiments of the present technology, the explicitassociation comprises at least one of: the first user subscribing to thecontent channel, the user liking a prior content item from the contentchannel, and the user commenting on the prior content item from thecontent channel.

In some non-limiting embodiments of the present technology, the givenitem of the set of recommended items is originating from a networkresource accessible via the communication network (such as one of thefirst network resource 132, the second network resource 134 and theplurality of additional network resources 136).

In some non-limiting embodiments of the present technology, the givenitem of the set of recommended items is one of a news article, an image,a video, and an interactive snippet.

In some non-limiting embodiments of the present technology, all items ofthe set of recommended items are not originating from the contentchannel. In some non-limiting embodiments of the present technology,another given item of the set of recommended items is originating from acontent channel being one of the content channel and another contentchannel. In some non-limiting embodiments of the present technology, thecontent channel is a native channel to the recommendation system. Assuch, even though embodiments of the present technology are particularlyapplicable to native content items, they are not so limited; and can beused for both native and non-native digital content items.

In some non-limiting embodiments of the present technology, theidentifying the pool of users associated with the content channelcomprises identifying the given user of the pool of users as beingassociated with the content channel based on at least one of: (i) animplicit association; and (ii) an explicit association.

It should be expressly understood that not all technical effectsmentioned herein need to be enjoyed in each and every embodiment of thepresent technology. For example, embodiments of the present technologymay be implemented without the user enjoying some of these technicaleffects, while other embodiments may be implemented with the userenjoying other technical effects or none at all.

Some of these steps and signal sending-receiving are well known in theart and, as such, have been omitted in certain portions of thisdescription for the sake of simplicity. The signals can be sent-receivedusing optical means (such as a fibre-optic connection), electronic means(such as using wired or wireless connection), and mechanical means (suchas pressure-based, temperature based or any other suitable physicalparameter based).

Modifications and improvements to the above-described implementations ofthe present technology may become apparent to those skilled in the art.The foregoing description is intended to be exemplary rather thanlimiting. The scope of the present technology is therefore intended tobe limited solely by the scope of the appended claims.

1. A method of determining a relevancy parameter for a digital contentitem, the digital content item being originated from a content channelassociated with a recommendation system, the relevancy parameter forranking the digital content item as a recommended content item for usersof the recommendation system, the recommendation system including aserver and at least one client device connectable to the server via acommunication network, the method executable by the server, the serverfurther being configured to execute a recommendation algorithm togenerate a set of recommended content items for a given user of therecommendation system; the method comprising: identifying a pool ofusers associated with the content channel, a given user of the pool ofusers being associated with the content channel; in response toreceiving a content recommendation request from a first client deviceassociated with a first user that belongs to the pool of users:generating, using the recommendation algorithm, the set of recommendedcontent items for the first user, a given item of the set of recommendedcontent items not originating from the content channel; artificiallyinserting into the set of recommended items the digital content item;gathering an indication of user interactions of the first user with theset of recommended items, the user interactions indicative of the firstuser's propensity for the digital content item; and predicting relevancyparameter of the digital content item for a user outside of the pool ofusers based on the user interactions of the first user.
 2. The method ofclaim 1, the method further comprises: in response to receiving acontent recommendation requests from other client devices associatedwith other users that belong to the pool of users: generating, using therecommendation algorithm, a respective set of recommended items for theother users, a given item of the respective set of recommended items notoriginating from the content channel; artificially inserting into therespective set of recommended items the digital content item.
 3. Themethod of claim 2, further comprising observing user interactions of theother users with the respective sets of content recommendations andgenerating an augmented relevancy parameter associated with the digitalcontent item based on the user interactions of the other users with therespective sets of content recommendations.
 4. The method of claim 3,wherein the predicting relevancy parameter of the digital content itemfor the user outside of the pool of users comprises: predicting therelevancy parameter based at least in part on the augmented relevancyparameter.
 5. The method of claim 4, wherein the augmented relevancyparameter is upwardly biased relative to a native relevancy parameterthat would be generated by the recommendation algorithm.
 6. The methodof claim 1, wherein the method further comprises: in response toreceiving a content recommendation request from a second client deviceassociated with a second user that is outside the pool of users:generating, using the recommendation algorithm, the set of recommendeditems for the second user, the set of recommended items including thedigital content item, inclusion of the digital content item into the setof recommended items being based on the relevancy parameter; gatheringan indication of user interactions of the second user with the set ofrecommended items, the user interactions indicative of the second user'spropensity for the digital content item.
 7. The method of claim 6,wherein in response to the user interactions of the second user beingindicative of lower propensity for the digital content item of thesecond user when compared to the first user: adjusting the relevancyparameter of the digital content item to a lower value thereof.
 8. Themethod of claim 1, wherein the artificially inserting the digitalcontent item into the set of recommended items comprises: ranking thedigital content item relative to other content items within the set ofrecommended items, the ranking being based on the predicted relevancyparameter of the digital content item.
 9. The method of claim 1, whereinthe artificially inserting the digital content item into the set ofrecommended items comprises: positioning the digital content itemrelative to other content items within the set of recommended items at apre-determined position within the set of recommended items.
 10. Themethod of claim 9, wherein the pre-determined position is selected suchthat to maximize a probability of the user interaction with the digitalcontent item.
 11. The method of claim 1, wherein the identifying thepool of users associated with the content channel comprises identifyingthe given user of the pool of users as being associated with the contentchannel based on at least one of: (i) an implicit association; and (ii)an explicit association.
 12. The method of claim 11, wherein theimplicit association comprises the first user having been presented witha prior content item from the content channel and the first user notproviding an indication of negative propensity in response thereto. 13.The method of claim 11, wherein the explicit association comprises atleast one of: the first user subscribing to the content channel, theuser liking a prior content item from the content channel, and the usercommenting on the prior content item from the content channel.
 14. Themethod of claim 1, wherein the given item of the set of recommendeditems is originating from a network resource accessible via thecommunication network.
 15. The method of claim 13, wherein the givenitem of the set of recommended items is one of a news article, an image,a video, and an interactive snippet.
 16. The method of claim 1, whereinall items of the set of recommended items are not originating from thecontent channel.
 17. The method of claim 1, another given item of theset of recommended items is originating from a content channel being oneof the content channel and another content channel.
 18. The method ofclaim 1, wherein the content channel is a native channel to therecommendation system.
 19. A recommendation server for generating adigital content recommendation, the digital content recommendation fordisplaying on an electronic device associated with a user, the serverconnectable to the electronic device via a communication network, therecommendation server executing a ranking algorithm, the recommendationserver comprising a processor configured to determine a relevancyparameter for a content item, the digital content item being originatedfrom a content channel associated with the recommendation server, therelevancy parameter for ranking the digital content item as arecommended content item for users of the recommendation server, therecommendation server including a server and at least one client deviceconnectable to the server via a communication network, the methodexecutable by the recommendation server, the processor of therecommendation server further being configured to execute arecommendation algorithm to generate a set of recommended content itemsfor a given user of the recommendation server; the processor beingfurther configured to: identify a pool of users associated with thecontent channel, a given user of the pool of users being associated withthe content channel; in response to receiving a content recommendationrequest from a first client device associated with a first user thatbelongs to the pool of users: generate, using the recommendationalgorithm, the set of recommended content items for the first user, agiven item of the set of recommended content items not originating fromthe content channel; artificially insert into the set of recommendeditems the digital content item; gather an indication of userinteractions of the first user with the set of recommended items, theuser interactions indicative of the first user's propensity for thedigital content item; and predict a relevancy parameter of the digitalcontent item for a user outside of the pool of users based on the userinteractions of the first user.